An information theoretic approach to statistical dependence: copula information

We discuss the connection between information and copula theories by showing that a copula can be employed to decompose the information content of a multivariate distribution into marginal and dependence components, with the latter quantified by the mutual information. We define the information excess as a measure of deviation from a maximum entropy distribution. The idea of marginal invariant dependence measures is also discussed and used to show that empirical linear correlation underestimates the amplitude of the actual correlation in the case of non-Gaussian marginals. The mutual information is shown to provide an upper bound for the asymptotic empirical log-likelihood of a copula. An analytical expression for the information excess of T-copulas is provided, allowing for simple model identification within this family. We illustrate the framework in a financial data set. Introduction. – Modeling statistical dependence has a pervasive role in science. Information theory provides a unifying framework for ideas from areas as diverse as differential geometry [1], physics [2–4], statistics and telecommunications [5]. From the information theoretic point of view dependence can be quantified by measuring the distance between a given model defined by a joint probability density φ(x) and a mean field model defined by φ0 = ∏N j=1 fj(xj), where fj(xj) are marginal densities fj(xj) = ∫ ∏ k 6=j dxk φ(x) [6]. The relative entropy given by